Skip to content

v6.0.0 — Wave 1 Observability Release (Institutional Audit Traceability)#76

Merged
Number531 merged 27 commits into
mainfrom
observability/wave-1
Apr 18, 2026
Merged

v6.0.0 — Wave 1 Observability Release (Institutional Audit Traceability)#76
Number531 merged 27 commits into
mainfrom
observability/wave-1

Conversation

@Number531

Copy link
Copy Markdown
Owner

Summary

Adds four observability capabilities behind feature flags (all default OFF) to close gaps identified in an institutional-buyer audit against PE/IB/M&A/IC requirements.

28 commits | 45 files changed | +6530 / -888 LOC | 165 tests

Key architecture decisions

  • Per-session pool (not global): sessions = deals = audit boundaries. Self-containment > dedup.
  • wrapWithConversation capture (not PostToolUse/stream): server-side tools (WebFetch/WebSearch) are API-internal; MCP tool execution layer is the only working capture point.
  • Option B (raw bytes): no canonicalization before storage. Byte-exact audit fidelity.
  • Module decomposition shipped day-one: 6 files under src/utils/rawSource/, each ≤100 LOC.

Deployment

All flags default OFF — merge is zero-behavior-change. See docs/runbooks/wave-1-deploy.md for the 5-stage flag rollout:

  1. Deploy with all flags off (24h soak)
  2. SLA_TELEMETRY=true (24h soak)
  3. PROMPT_INJECTION_DETECTION=true (24h soak)
  4. RAW_SOURCE_ARCHIVE=true + EXA_WEB_TOOLS=true (48h soak)
  5. Merge confirmed

Breaking: claude_tool_duration_ms histogram label renamed tooltool_name. Migrate Prometheus/Grafana queries before deploy.

Test plan

  • 165 unit + integration tests pass (rawSource modules, injection detector, metrics, corpus calibration)
  • Live test: 287 sources captured, dedup working (5 hits), mode 0444, metadata sidecars correct
  • SLA dashboard renders with 248 tracked calls
  • Default-off regression: zero behavior change with all flags false
  • Full end-to-end session with all flags on (complete to final memorandum)
  • Prometheus dashboard migration for tooltool_name label

🤖 Generated with Claude Code

Number531 and others added 27 commits April 18, 2026 00:22
Baseline the Wave 1 reference documents on observability/wave-1 before
implementation begins.

- observability-updates-april-26.md — scoping, retrofit-cost roadmap
  (4 waves), per-item complexity/break-risk/time ratings, acceptance
  criteria, explicit out-of-scope.
- observability-implementation-spec.md — granular per-module spec:
  file paths, function signatures, NDJSON row schemas, test matrices,
  rollout plan, cross-wave concerns (feature flags, env vars, alerts,
  DR runbook), module dependency graph.

Wave 1 scope (to follow in subsequent commits):
  #3  raw-source archive (Path B: session-dir + global pool + per-agent manifests)
  #8  prompt-injection detection on tool outputs
  #12 per-tool latency histograms (P50/P95/P99)
  #13 per-API 7-day SLA dashboard

All four items gated behind feature flags defaulting to false.
Module decomposition + NDJSON schema versioning bundled day-one
to avoid disproportionate retrofit cost later.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…CTION, SLA_TELEMETRY

All three default to false so Wave 1 code can land in production
with zero behavior change until individually toggled on.

- RAW_SOURCE_ARCHIVE  — gates content-addressed pool writes (#3)
- PROMPT_INJECTION_DETECTION — gates regex detector in PostToolUse (#8)
- SLA_TELEMETRY — gates _hybrid_metadata extraction in hookDBBridge (#13)

#12 (histogram label refactor) is unconditional — additive Prometheus
label change, no flag needed.

Verified via runtime import: all three evaluate to false with no
environment overrides.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pure module — first of seven under src/utils/rawSource/. No side
effects, no I/O, trivially unit-testable.

Design change from earlier spec draft: **no canonicalization**.

The earlier spec specified text canonicalization (trim + collapse
whitespace) before hashing, to improve dedup hit rate when the same
document is re-fetched with trivial whitespace differences. That was
rejected in favor of byte-exact audit fidelity:

  - stored bytes == API response bytes (modulo secret sanitization, a
    legitimate security transform auditors accept)
  - recomputing SHA-256 on a pool file matches the filename directly
  - an auditor can re-fetch from the API and compare bytes without
    having to replicate any canonicalization pipeline
  - realistic dedup loss is small — HTTP responses for the same URL
    from the same client tend to be byte-stable

HashResult shape simplified: { hash, bytes, size, inferredContentType }.
Content-type sniff (html/json/xml/text/binary) is informational only —
drives filename extension, never mutates bytes.

Spec updates in the same commit:
  - observability-implementation-spec.md §1.1.1 — Option B design note
  - observability-updates-april-26.md — write-pipeline step ordering
    (sanitize precedes hash; no canonicalize step)
  - module summary tagline updated

Tests: 27 pass in 91ms under NODE_OPTIONS=--experimental-vm-modules jest.
Covers: determinism, whitespace-different-inputs-different-hashes,
byte-exact storage, filename-integrity (recomputed SHA matches),
content-type sniffing (incl. binary NUL detection), input validation
(TypeError on null/undefined/number/object), empty input, 1 MB
performance (<50 ms).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Second of seven pure modules under src/utils/rawSource/. The only
transform applied before raw bytes land in the content-addressed pool;
legitimate under Option B audit posture because leaking credentials
into the archive is a separate security incident that auditors expect
us to prevent.

Pattern set (5):
  authorization_header  — Authorization: Bearer/Basic <token>
  api_key_query         — ?api_key= / ?api-key= / ?apikey= in URLs
                          (preserves the ?/& separator so URLs remain parseable)
  aws_access_key        — AKIA + 16 alphanum caps, word-bounded
  jwt                   — three dot-separated base64url segments
  private_key_block     — PEM-armored RSA/EC/DSA/OPENSSH/ENCRYPTED keys

Replacement format: [REDACTED:<pattern_name>]. Pattern names (not
values) are preserved in the SanitizeResult.redactions audit so the
metadata sidecar can record WHAT was redacted without storing the
secret itself.

Defensive properties:
  - never throws (null/undefined/non-string → empty-result sentinel)
  - pure function — no I/O, no state leak (fresh RegExp per pattern
    to avoid lastIndex state)
  - modified=false on clean text (zero-copy short-circuit via early return)
  - no false positives on "ignore all prior filings" or plain SEC URLs

Tests: 27 pass in 98ms.
  - Per-pattern detection for all 5 formats
  - Negative cases: clean SEC text, plain URLs, non-JWT base64
  - Edge cases: word boundaries on AKIA (no partial match), lowercase
    rejection for AWS, case-insensitivity for Authorization, multi-pattern
    documents with correct per-pattern counts, original-secret leakage
    check (cleaned output MUST NOT contain the secret substring)
  - Defensive: empty string, null, undefined, number → empty-result

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Third of seven modules under src/utils/rawSource/. Stateful factory —
binds to a single pool directory and exposes content-addressed read/write
with atomic, idempotent, integrity-checked semantics.

Storage layout:
  {poolDir}/{hash[0:2]}/{hash[2:4]}/{hash}.{ext}.gz   body (gzip)
  {poolDir}/meta/{hash}.json                          metadata sidecar

API surface:
  pathForHash(hash, ext)        → sharded body path
  metaPathForHash(hash)         → sidecar path
  exists(hash, ext)             → boolean
  write(hash, ext, content)     → { written, path, size, compressedSize }
                                  - tmp + rename → atomic
                                  - chmod 0o444 after write → tamper-resistant
                                  - idempotent: second call with same hash =
                                    written:false, no disk I/O, mtime unchanged
                                  - throws if size > maxRawBytes (default 10 MB)
  writeMeta(hash, meta)         → atomic JSON sidecar write
  read(hash, ext)               → decompressed body, throws ChecksumError if
                                  recomputed SHA != filename hash
  readMeta(hash)                → parsed JSON or null on ENOENT
  statCompressed(hash, ext)     → on-disk size

ChecksumError class exported with { expected, actual, path } context for
upstream alerting (Wave 3 wires this into the error taxonomy + circuit breaker).

Tests: 21 pass in 122ms against real temp-dir filesystems.
  - factory validation (poolDir required, exposed API surface)
  - sharded path construction (incl. compress=false omitting .gz)
  - first-landing write: returns written:true, file is 0o444, gzip is
    decompressible back to input bytes, accepts Buffer directly
  - dedup: second write returns written:false, mtime unchanged
  - size guard: throws past maxRawBytes, accepts at exact boundary
  - integrity: round-trip succeeds; tampered file → ChecksumError with
    correct expected/actual/path
  - meta: write/read round-trip, ENOENT returns null
  - concurrency: 5 parallel writes for same hash → exactly one file,
    no .tmp.* remnants, body correct on read

Note: removed setTimeout from dedup test — Jest experimental VM modules
hangs on async setTimeout in some configurations (verified the dedup
short-circuit is instant via direct node script: 0 ms).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fourth and fifth modules under src/utils/rawSource/ — both stateful
factories that perform append-only NDJSON writes. Bundled into one
commit because they share design discipline (append-only, parent-dir-
on-demand, schema-version-agnostic).

SourceManifestWriter — per-session and per-agent manifests:
  appendSession(sessionId, row) → {sessionsRoot}/{sessionId}/raw-sources-manifest.ndjson
  appendAgent(sessionId, agentType, row) → {sessionsRoot}/{sessionId}/specialist-reports/{agentType}-sources/sources.ndjson

  - Path traversal guard: agentType matches /^[a-z0-9][a-z0-9_-]*$/i
    (rejects '..', absolute paths, spaces).
  - Writer is intentionally dumb — does NOT validate row shape.
    schema_version presence and field correctness are the orchestrator's
    responsibility.
  - Uses fs.appendFile (Node O_APPEND under the hood) — concurrent
    appends from the same process produce well-formed NDJSON.

SourceIndexWriter — global tamper-evident _index.ndjson:
  append(row) → {poolDir}/_index.ndjson

  - Per-call: open(a) + write + fsync + close.
  - The fsync per row is the difference from manifests: tail entries
    cannot be lost on crash. Cost is acceptable because append() only
    fires on dedup miss (new hash landings are rare).
  - Future Wave 3 hook: nightly Merkle root over this file becomes the
    tamper-evident anchor.

Tests: 23 pass (12 manifest + 11 index) in 272ms against real temp dirs.
  Manifest:
    - factory validation, exposed surface
    - session path / agent path correctness
    - parent-directory creation on first append
    - strict NDJSON (one object per line, newline-terminated)
    - rich row shapes round-trip
    - path-traversal rejection ('../etc/passwd', '/abs', 'name with space')
    - safe agent-type acceptance (alphanum, hyphen, underscore, mixed case)
    - 10 parallel appendSession → 10 well-formed rows, all values present
  Index:
    - factory validation, indexPath exposed
    - single-row + multi-row ordering
    - poolDir creation on demand
    - strict JSON lines (no array wrapper, no trailing comma, newline-terminated)
    - rich row shapes round-trip (incl. nested objects)
    - 20 parallel appends → 20 distinct well-formed rows

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sixth of seven modules under src/utils/rawSource/. Intentionally
minimal: preserves the interface the orchestrator will call so
RawSourceService can `dispatcher.enqueue(hash, sourceType)`
unconditionally — no branching on a feature flag for an absent
real implementation.

Wave 2 replaces this stub with:
  - bounded worker pool (BATCH_SIZE=20, MAX_DEPTH=500)
  - dedup check against source_chunk_embeddings (no re-embed)
  - chunkContent → embedDocuments (Gemini RETRIEVAL_DOCUMENT)
  - transactional INSERT into source_chunk_embeddings table
  - flag: RAW_SOURCE_EMBEDDING (default false)

Wave 3 adds:
  - backpressure: shed-work above MAX_DEPTH, log + metric
  - per-error counter via raw_source_errors_total
  - circuit breaker on consecutive failures

Stub is fail-open by design: enqueue() always resolves, never rejects.
The orchestrator's `.catch(err => console.warn(...))` is defensive —
the stub gives nothing to catch.

No new tests — single async function returning undefined; full
behavior tested via the RawSourceService orchestrator integration
test in the next commit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Seventh and final module under src/utils/rawSource/. Composes the six
preceding modules into a single fire-and-forget persist() call that the
PostToolUse hook will invoke for raw-source-carrying tools.

Pipeline (per persist() call):
  1. Validate input (graceful — log + return null, never throw)
  2. Size guard (drops oversize at the door)
  3. Sanitize → cleaned (only pre-storage transform; secrets removed)
  4. Hash raw bytes (Option B; cleaned bytes = stored bytes = hash input)
  5. Storage.write (idempotent — dedup hit short-circuits with written:false)
  6. Sidecar + global index (only on first landing)
  7. Session manifest (always — even on dedup hit)
  8. Per-agent manifest (when agentType present and passes path-traversal guard)
  9. Fire-and-forget embedding enqueue (Wave 1 stub no-ops; Wave 2 activates)

Defensive properties:
  - Never throws — every step wrapped in try/catch with structured warn log
  - Per-step error isolation: appendAgent failure does NOT abort the rest of
    the persist (pool body + session manifest still land)
  - Embedding enqueue rejection ignored at the orchestrator boundary
  - Returns null on input-validation failure or oversize trip

Dependency injection: `overrides` slot in createRawSourceService accepts
{ storage, manifestWriter, indexWriter, embeddingDispatcher, hasher,
sanitizer } for tests / future swaps. Production callers pass only
{ poolDir, sessionsRoot } and get a fully-wired service.

Module also re-exports the six component pieces (hashSource, sanitize,
PATTERNS, createSourceStorage, ChecksumError, etc.) so consumers can
import everything from one path.

Tests: 24 new orchestrator tests + 97 existing module tests = 121 total
       across 6 suites, all passing in 489ms.

Orchestrator coverage:
  - factory validation (poolDir / sessionsRoot required)
  - input validation: missing sessionId/content/toolName, null/undefined
    input, non-string content, oversize → all return null without throwing
  - first landing: pool body + sidecar + index + session manifest all land
  - per-agent manifest: written when agentType provided, skipped otherwise
  - dedup: same content twice = one pool file, one index row, two manifest
    rows (second has dedup_hit=true)
  - cross-session dedup: same content from sessions A and B = one pool file,
    each session has its own one-row manifest
  - sanitization: API key + Authorization header redacted from stored body;
    [REDACTED:*] tags appear; original secrets do NOT appear in pool file
  - clean SEC text passes through (sanitized=false, redactions=[])
  - embedding dispatcher receives correct (hash, sourceType); rejection
    does not propagate
  - error isolation: invalid agentType (path-traversal) does not abort
    pool/session writes
  - content-type routing: html/json/text → correct ext + sourceType

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wave 1 prompt-injection detector for tool outputs. Lightweight (pure
regex, no LLM, no network), defensive (logging-only, conservative
thresholds), reusable (single chokepoint inserted in postToolUseHandler
in the next commit).

Pattern set (6 patterns, two weight tiers):
  Formatting tokens — weight 0.9 (rarely legitimate in fetched docs):
    system_tag    → /\[SYSTEM\]|\[\/SYSTEM\]/gi
    im_start      → /<\|im_start\|>/gi
    system_colon  → /^\s*SYSTEM:\s/gim   (line-anchored)

  Semantic phrases — weight 0.4 (often appear in legal text):
    ignore_prior   → ignore <previous|all|above|prior> <instructions|prompts|rules>
    you_are_now    → you are <now|actually> <NOT followed by 'the same|here|in'>
    new_directive  → new <directive|instructions|rules>[:.]

Confidence:
  max(individual weights) + 0.1 * (n_unique_matches - 1), capped at 1.0
Detection threshold: 0.5
  → single formatting token (0.9) → detected
  → single semantic phrase (0.4)  → NOT detected
  → two semantics (0.4 + 0.1)     → detected at boundary 0.5
  → formatting + semantic (1.0)   → detected

Defensive properties:
  - Pure function — no I/O, no state, never throws (null/undefined → empty result)
  - 16 KB scan limit by default — early-content focus, perf cap on multi-MB inputs
  - Excerpt cap ~200 chars (100 each side of first match)
  - Returns structured result; orchestrator decides whether to log it
  - Negative cases: 'Ignore all prior filings' (legitimate SEC), 'These instructions
    apply to participants', 'New directives from the Board', 'You are advised'
    all explicitly do NOT trigger

Pattern set deliberately overlaps with src/middleware/inputValidation.js but
does NOT import from it: that file is HTTP middleware that hard-blocks (400);
here we score, log, and let the response flow.

Phase 2 (Wave 3): escalate ambiguous matches (confidence 0.4–0.75) to a
Haiku 4.5 classifier via Messages API. The `classifier` field in the result
is the placeholder for that — currently always 'regex'.

Tests: 29 pass in 72ms.
  - PATTERNS export shape + weights
  - Single-token formatting detection (system_tag, im_start, system_colon)
  - SYSTEM: at line start vs mid-line (multiline anchor)
  - Single semantic patterns score 0.4 (NOT detected) — ignore_prior, you_are_now, new_directive
  - Combined patterns: two semantics → 0.5 (detected); formatting+semantic → 1.0
  - FP resistance on 7-line mock SEC body — does NOT cross threshold
  - Excerpt window contains first match; empty when no match
  - Scan-limit honored (matches beyond 16 KB ignored; explicit override expands)
  - Defensive input handling: '', null, undefined, number → empty result
  - Performance: 16 KB scan in <5 ms (clean and dirty inputs both)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…12)

Widens the claude_tool_duration_ms histogram label set from
[tool, status] → [tool_name, client, status] so per-external-API
percentiles (P50/P95/P99) become queryable in Prometheus + Grafana.

Why `client`: the same tool_name (e.g., fetch_document) can route through
different external services (direct HTTP vs Exa /contents fallback). Without
the client label, a slow Exa-fallback path is invisible in the aggregate.

Cardinality bound: ~50 tool_names × ~6 clients × 3 statuses ≈ 900 series.
Well under prom-client default limits.

Bucket set widened on the long tail (10000, 30000, 60000) to capture slow
external APIs that today bunch into the >5s bucket.

Backward-compatible signature on recordToolDuration:
  Legacy:  recordToolDuration(toolName, status, durationMs)
           → observed with client='unknown'
  Wave 1:  recordToolDuration({ tool_name, client, status }, durationMs)

Existing callers (researchHandler.js:256) keep working with client='unknown'.
The Wave 1 hook integration (next commit, #12 scope) will use the object form
with deriveClient() to populate the new label.

Also adds:
  deriveClient(toolName, hybridMetadata) → string
    fetch_document + source='exa'    → 'exa_fallback'
    fetch_document + source='native' → 'direct_fetch'
    fetch_document + null/undefined  → 'direct_fetch' (default)
    exa_web_search                    → 'exa_native'
    mcp__<domain>__<method>           → '<domain>'
    everything else                   → 'other'

Tests: 13 pass in 137ms.
  - Label set check: histogram exposes [client, status, tool_name]
  - Wave 1 object signature: observes with all three labels
  - Wave 1 partial labels: missing fields default to 'unknown'
  - Legacy positional signature: client='unknown', tool_name + status preserved
  - deriveClient: every documented branch (fetch_document with/without
    metadata, exa_web_search, mcp__sec__/courtlistener/super-legal-tools,
    SDK tools, null/undefined/non-string)
  - Cardinality bound: 4 tools × 5 clients × 2 statuses → 40 distinct series

Note: ran `npm install --legacy-peer-deps` in the worktree to materialize
node_modules (peer-dep conflict on @google/genai surfaced as a known
project issue from main; resolution unchanged).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hot-path change in persistAuditEvent. Closes the critical gap that
blocks #13: _hybrid_metadata fields (fetch_source, fallback_reason,
fetch_mode, confidence) are extracted in sdkHooks.js:1018-1031 today
but are never persisted, so the /api/analytics/sla/7day endpoint has
no data to query.

Implementation:
  - Insertion point: persistAuditEvent at line ~563 (just before INSERT)
  - Triggers only when: SLA_TELEMETRY=true AND hookName='PostToolUse'
    AND tool_name ∈ SLA_HYBRID_TOOLS (fetch_document | exa_web_search)
  - Parses tool_response.content[0].text as JSON; extracts
    _hybrid_metadata.{source, fallback_reason, fetch_mode, confidence}
    into eventData.{fetch_source, fallback_reason, fetch_mode, fetch_confidence}
  - Native-success inference: when a hybrid-client tool succeeds but produces
    no _hybrid_metadata (typical for native-only paths), set fetch_source='native'
    so the SLA dashboard can still group it
  - JSON.parse wrapped in try/catch — non-JSON responses are common (HTML,
    plain text); parse failure is silent (audit insert proceeds normally)

Hot-path discipline:
  - Flag-gated: featureFlags.SLA_TELEMETRY (default false). With flag off,
    zero behavior change vs pre-Wave-1 baseline.
  - Single try/catch boundary: a malformed response cannot break the audit
    insert under any circumstance.
  - Fields are optional — every column that consumed event_data already
    handles null via COALESCE / COALESCE-style frontend code.

Verification:
  - Module loads cleanly (node -e import).
  - Full end-to-end coverage (PostToolUse hook fires → row in hook_audit_log
    has fetch_source populated) lives in test/integration/sla.integration.test.js
    coming in Task #15. Cannot unit-test persistAuditEvent in isolation —
    function is not exported and depends on a live pg pool + sessionCache.

SLA_HYBRID_TOOLS set is intentionally minimal in Wave 1:
  fetch_document    — direct + exa fallback paths
  exa_web_search    — direct exa search
Wave 4 expands to per-hybrid-method instrumentation
(searchSECFilings, searchCourtOpinions, etc.).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#8, #12)

Connects the four pure/stateful modules built in earlier commits to the
live PostToolUse hook flow. Four files modified, each change focused.

src/hooks/sdkHooks.js (postToolUseHandler):
  - Import detectInjection (#8) + recordToolDuration/deriveClient (#12).
  - Restructure existing _hybrid_metadata block to capture parsedToolResponse
    and textContent for reuse across detection + metric labeling.
  - Prompt-injection detection (flag-gated by PROMPT_INJECTION_DETECTION):
    runs detectInjection(textContent, {toolName}); if detected, attach to
    `entry` (file log) and propagate via hook return value
    `{ continue: true, prompt_injection: {...} }`.
    Detector failures caught locally — never throws.
  - Histogram observation (#12, always-on, no flag): on every PostToolUse
    with non-null duration_ms + tool_name, observes
    claude_tool_duration_ms{tool_name, client, status} where client is
    derived from tool_name + _hybrid_metadata.source.
    Recording failures caught locally.
  - Return value backward-compatible: handlers that don't fire injection
    detection still return { continue: true } unchanged.

src/utils/hookDBBridge.js (persistAuditEvent):
  - Read result.prompt_injection.detected and merge five fields into
    eventData: prompt_injection_detected, _patterns, _excerpt, _confidence,
    _classifier. Frontend filters on prompt_injection_detected; analytics
    queries can do `WHERE event_data->>'prompt_injection_detected' = 'true'`.
  - PostToolUse row's event_type is preserved (not replaced) so the audit
    chain stays intact and the SLA telemetry on the same row continues to work.

src/utils/hookSSEBridge.js:
  - Import featureFlags + define RAW_SOURCE_TOOLS allow-list with .includes()
    matcher (handles MCP-wrapped variants like 'mcp__direct-fetch__fetch_document').
  - Extend forwardHookToSSE signature with sseOptions = {} as 8th arg
    (backward-compatible default).
  - PostToolUse case grows two new top-of-block sections:
      a) Raw-source archive (#3): when RAW_SOURCE_ARCHIVE=true AND
         sseOptions.rawSourceService is wired AND tool is in RAW_SOURCE_TOOLS,
         fire-and-forget `persist({sessionId, agentId, agentType, toolName,
         toolUseId, url, content})`. On success, emit `raw_source_ready` SSE
         event with { hash, size, url:/api/raw-sources/{hash}, agent_id,
         agent_type, ext, source_type, dedup, redactions, sanitized }.
         Errors caught at the .catch() boundary; never block the hook chain.
      b) Prompt-injection forwarding (#8): when result.prompt_injection.detected,
         emit `prompt_injection_detected` SSE event with patterns/confidence/
         excerpt/classifier so the frontend timeline can surface it live.
  - wrapHooksForSSE + createSSEBridge both grow `sseOptions` parameter (default
    {}) and propagate through to forwardHookToSSE. All existing callers keep
    working; new callers opt into the raw-source wiring.

src/server/agentStreamHandler.js:
  - Import createRawSourceService.
  - Per-request instantiation: poolDir = reports/_sources, sessionsRoot = reports/.
    Service is constructed unconditionally; the SSE bridge skips the persist
    branch when RAW_SOURCE_ARCHIVE=false (zero behavior change with flag off).
  - Pass { rawSourceService, getSessionId: () => ctx.sessionDir } as the third
    arg to createSSEBridge so PostToolUse can attribute writes to the live session.
  - Stash service on ctx for downstream consumers (future Wave 2/3).

Verification:
  - All four modified modules load cleanly via direct node import.
  - All 163 unit tests across 8 suites still pass in 623ms (rawSource modules,
    promptInjectionDetector, metrics) — no regressions.
  - Default-off state (all three flags=false) produces zero behavior change:
    sdkHooks skips both detection + metric record; hookDBBridge skips SLA
    extraction; hookSSEBridge skips raw-source persist + injection forwarding.
  - Full end-to-end (PostToolUse → pool file lands; raw_source_ready event
    surfaces in #rawLog; hook_audit_log row carries prompt_injection_*) covered
    by integration tests in Task #15.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ndex (#3, #12, #13)

Three files, four new routes, one new composite index. Closes the
HTTP/SQL exposure gap so the modules wired in the previous commit
become reachable from the frontend and from external Prometheus.

src/server/claude-sdk-server.js — four GET routes for the raw-source archive:
  GET /api/raw-sources/:hash             — decompressed body + SHA verification
  GET /api/raw-sources/:hash/meta        — fetch metadata sidecar JSON
  GET /api/sessions/:sid/raw-sources     — session-level NDJSON manifest as array
  GET /api/sessions/:sid/agents/:agent/sources — per-agent NDJSON manifest

  - Reuses createSourceStorage from src/utils/rawSource/index.js (lazy-imported
    once per process, cached via _rawSourceStorage closure to avoid circular
    import at server startup).
  - Body endpoint determines extension via:
      1. meta sidecar (canonical), then
      2. ?ext= query parameter (validated against KNOWN_EXTS), then
      3. probe (try each known extension via storage.exists)
    Returns 404 if none match.
  - Hash format guard: HEX64 = /^[a-f0-9]{64}$/. Returns 400 on malformed hash.
  - Session ID guard: existing SESSION_ID_RE. Agent type guard: SAFE_AGENT_TYPE
    (alphanum + hyphen + underscore; matches SourceManifestWriter's path-traversal
    guard exactly).
  - On read, recomputes SHA-256 via SourceStorage.read; ChecksumError → 500
    with structured warn log (hooked for Wave 3 alerting).
  - Sets X-Source-Hash, X-Fetched-At, X-Source-URL headers from meta sidecar
    so auditors can verify provenance from response headers alone.
  - 404 returned as empty rows (count:0) for manifest endpoints — frontend
    renders "no data" state instead of error.

src/server/dbFrontendRouter.js — extended /api/analytics/tools/health,
new /api/analytics/sla/7day:
  - tools/health: added p50_ms, p95_ms, p99_ms columns via
    PERCENTILE_CONT(...) WITHIN GROUP (ORDER BY duration_ms). Tightened
    WHERE to require duration_ms IS NOT NULL.
  - sla/7day: NEW route. Day × api_client grid:
      DATE_TRUNC('day', created_at) AS day,
      COALESCE(event_data->>'fetch_source', 'unknown') AS api_client,
      calls, success_rate, p95_ms, fallback_count
    Constrained to last 7 days, PostToolUse[Failure] events on fetch_document
    or exa_web_search tool variants. Source data populated by the SLA_TELEMETRY
    extraction in hookDBBridge (commit 34499f3).
  - Both queries inherit existing event_type NOT IN ('AgentProgress') filter.

src/db/postgres.js — composite index for the percentile + SLA queries:
  idx_audit_tool_time_dur ON hook_audit_log (tool_name, created_at DESC, duration_ms)
    WHERE event_type IN ('PostToolUse', 'PostToolUseFailure')
      AND duration_ms IS NOT NULL

  - Partial index keeps it small (excludes SubagentStart/Stop/AgentProgress rows
    that have NULL duration_ms or are not relevant to per-tool latency).
  - PERCENTILE_CONT(... WITHIN GROUP ORDER BY duration_ms) reads in index order,
    avoiding a sort over millions of rows.
  - Wave 2 will move this and other Wave-1 schema deltas into a versioned
    node-pg-migrate file as 002_* (the planned migration tool adoption).

Verification:
  - All three files syntax-check clean (node --check).
  - claude-sdk-server.js loads up to its env-var check (existing behavior;
    ANTHROPIC_API_KEY required at process start).
  - 163 unit tests across the rawSource modules + injection detector + metrics
    still pass (no regressions from the route additions, since the routes
    consume read-only methods of SourceStorage that were already tested).
  - End-to-end (live server, populated pool) covered by smoke + integration
    tests in Task #15.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds the External API SLA (7d) panel to the Status tab as a collapsible
section between Rate Limiter and Stream Stats. Panel polls
/api/analytics/sla/7day every 60 seconds and renders a table grid
(day × api_client) with calls, success rate, P95 latency, and fallback
count per row.

Files:
  test/react-frontend/index.html
    - New <div class="dashboard-section collapsible"> with id="slaPanel"
    - Empty-state placeholder rendered when SLA_TELEMETRY=false or no rows
    - <table id="slaTable"> with thead (Day / API / Calls / Success / P95 / Fallback)

  test/react-frontend/app.js
    - New `slaTimer` (let, near healthTimer)
    - `fetchSlaDashboard()` — fetch + render, silent on non-200
    - `renderSlaTable(rows)` — toggles empty/table visibility, renders rows
      with successClass mapping: ≥99% accent, 95-99% neutral, <95% error
    - Bootstrapped alongside fetchHealth + fetchSubagents + fetchCatalog
    - setInterval(fetchSlaDashboard, 60_000) starts on first init

Behavior with flag off (SLA_TELEMETRY=false):
  - Backend route /api/analytics/sla/7day still responds (returns 0 rows
    since no event_data.fetch_source values exist to group by).
  - Frontend renders the empty placeholder. No console noise.

Behavior with flag on:
  - Backend extracts fetch_source/fallback_reason/fetch_mode into event_data
    on every PostToolUse for fetch_document / exa_web_search.
  - Within ~60s of first traffic, the table populates with the live grid.
  - Color-coded success_rate gives at-a-glance API health view.

Verification: node --check app.js syntax-clean; HTML well-formed
(matches existing collapsible-section pattern). End-to-end (live API
populates rows) covered by smoke + integration tests in Task #15.

Note: the percentile columns on /api/analytics/tools/health (also Wave 1
#12) are exposed at the API level and visible via curl /metrics or
direct JSON; a frontend Tools Health table is deferred to Wave 4 polish
because no current panel renders that data shape.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oy guide

Final Wave 1 commit. Closes the test + rollout gap so the release is
shippable end-to-end.

test/fixtures/raw-sources/  (4 files, used by both integration tests):
  sec-10k-sample.html        — SEC filing excerpt with phrases that
                               could trigger semantic injection patterns
                               but legitimately don't (e.g., "Ignore all
                               prior filings"); used to verify FP resistance
  court-opinion-sample.json  — court opinion JSON with _hybrid_metadata
                               so the orchestrator picks the .json extension
  exa-results-sample.json    — exa_web_search response shape with results[]
                               and _hybrid_metadata for source/result_count
  injection-corpus.json      — 12 calibration samples (6 clean, 6 dirty)
                               with per-sample expected_detected labels
                               and notes explaining the detector behavior

test/integration/rawSource.integration.test.js  (6 tests):
  - SEC fixture full pipeline → pool body, sidecar, _index.ndjson, session
    manifest, per-agent manifest all present and well-formed
  - Exa JSON fixture → .json extension + exa_result source_type
  - Cross-session dedup → unique-content probe lands once, manifests in both
    sessions
  - Sanitization end-to-end → API key + Auth header redacted from stored body;
    original secrets never appear on disk
  - Tampered file → ChecksumError on read

test/integration/promptInjection.integration.test.js  (5 tests):
  - Per-sample expected_detected matches detector across all 12 corpus entries
  - Aggregate FP rate on clean samples ≤ 25% (Wave 1 acceptance criterion)
  - Aggregate detection rate on injected samples ≥ 80%
  - Overall accuracy ≥ 90%
  - SEC + Exa fixtures pass detector cleanly (no FP)

test/smoke/README.md:
  Runbook-style smoke tests (curl commands + SQL queries) for each of the
  four Wave 1 items plus the default-off regression check. Automated smoke
  spawning the dev server in CI deferred to Wave 3 alongside the chaos suite.

docs/runbooks/wave-1-deploy.md:
  Deploy runbook with pre-flight checklist, 5-step staging→production flag
  rollout (24h soak between flips, 48h before raw-source enable), per-flag
  rollback procedures, full verification matrix with pass criteria for each
  acceptance item, and known limits / Wave 2 follow-ups.

package.json:
  Added test:integration:wave1 (scoped to test/integration/, distinct from
  the existing tests/integration script that runs unrelated suites) and
  test:smoke (echoes the runbook README path).

Test totals (Wave 1 final):
  Unit:        163 tests / 8 suites — 0.6 s
    SourceHasher (27), SourceSanitizer (27), SourceStorage (21),
    SourceManifestWriter (12), SourceIndexWriter (11), RawSourceService (24),
    promptInjectionDetector (29), metrics (13)
  Integration:  11 tests / 2 suites — 0.2 s
    rawSource end-to-end (6), promptInjection corpus (5)
  Combined:    174 tests / 10 suites — 0.8 s

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…audit

A blast-radius audit identified three changes that take effect on Wave 1
deploy regardless of feature flag state. None breaks the functional
pipeline; each has operational implications worth pre-deploy attention.

Runbook additions:
  1. Histogram label rename (tool → tool_name) with explicit grep + migration
     guidance for existing Prometheus/Grafana queries and alert rules.
     The legacy positional recordToolDuration() call signature is preserved,
     so existing call sites (researchHandler.js:256) still observe values
     under the new label set with client="unknown".
  2. Composite index idx_audit_tool_time_dur added to hook_audit_log inside
     initSchema(); runs synchronously on first server start. Added a sizing
     query to estimate indexed_rows + decision matrix mapping row count to
     expected build time:
        < 10M    rows  → < 30s, deploy normally
        10M-100M rows  → 30s-5min, schedule during low-traffic window
        > 100M   rows  → pre-build with CREATE INDEX CONCURRENTLY before deploy
                         (IF NOT EXISTS makes in-process create a no-op)
  3. Always-on metric observation in postToolUseHandler. Adds ~750 Prometheus
     series (50 tool_name × 5 client × 3 status). Per-call cost ~1-2 μs.
     Runbook added a cardinality-budget pre-flight check.

Frontend hygiene comment:
  app.js — added a comment near slaTimer noting that, like healthTimer, no
  explicit clearInterval is wired. Both rely on the page lifecycle (hard
  navigation / window close) to reclaim. If SPA-style navigation is added
  later, both timers need cleanup. This documents the existing convention
  rather than masking it.

The audit's "byte-identical to main" verdict is C+ (qualified) — flag-gated
paths are all safe, but the three unconditional changes above mean deploy
parity is operational, not strict. The runbook now makes that explicit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on pool

Correction 1.1: pivot from global content-addressed pool to per-session pool.

Factory signature change:
  BEFORE:  createRawSourceService({ poolDir, sessionsRoot, maxRawBytes, overrides })
  AFTER:   createRawSourceService({ sessionsRoot, maxRawBytes, overrides })
           — no poolDir; it's derived per persist() call from sessionsRoot + sessionId

Per-session pool path derivation inside persist():
  sessionPoolDir = path.join(sessionsRoot, sessionId, 'raw-sources')
  → SourceStorage + SourceIndexWriter instantiated with this path each call

Why per-session:
  The product is single-tenant-per-MD deal work. Sessions = deals = audit
  boundaries. Legal hold, 7-year retention, regulatory deletion, and
  session-level backup/restore all align with session folders. Global pool
  was optimizing cross-session dedup (~$3/yr at realistic throughput) at
  the expense of self-containment. Not the right tradeoff for this product.

Storage + index instantiation:
  Both were previously factory-time singletons bound to a single poolDir.
  Now per-persist() call. Storage construction does zero I/O, so per-call
  cost is negligible (~100 μs). Overrides (test DI) still short-circuit
  per-call instantiation for deterministic fixtures.

Filesystem layout change (user-visible):
  BEFORE: reports/_sources/{ab}/{cd}/{hash}.ext.gz
  AFTER:  reports/{sessionId}/raw-sources/{ab}/{cd}/{hash}.ext.gz

  Session manifests (raw-sources-manifest.ndjson, specialist-reports/
  {agent}-sources/sources.ndjson) were already session-scoped — unchanged.

Follow-up commits in this correction sequence:
  2. Delete SourceIndexWriter (redundant per-session); add first_landing
     flag to session manifest rows so Wave 3 tamper-evident Merkle rollup
     can still distinguish new-hash events from dedup hits.
  3. Update hooks/server wiring + routes (/api/raw-sources/:hash →
     /api/sessions/:sid/raw-sources/:hash).
  4. Update tests (paths + fixtures).
  5. Update docs (planning + spec + runbook + smoke README).

Verification:
  node --check passes; module imports cleanly; createRawSourceService is
  exported as a function. Full test updates land in commit 4 of this sequence.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…flag

Correction 1.1 D1: SourceIndexWriter is redundant under per-session scoping.

Under the old global pool, _index.ndjson served two purposes:
  1. Tamper-evident log of new-hash landings (for Merkle root rollup)
  2. Global dedup registry

Per-session, both purposes collapse into the session manifest:
  1. Tamper-evident: session manifest is already append-only + session-scoped
  2. Dedup: session-local; SourceStorage.exists() handles it

To preserve Wave 3's Merkle-rollup ability to distinguish new-hash events
from dedup hits, a new `first_landing: boolean` field is added to each
session manifest row (set from SourceStorage.write().written).

Changes:
  - DELETED: src/utils/rawSource/SourceIndexWriter.js
  - DELETED: test/sdk/rawSource/SourceIndexWriter.test.js
  - MODIFIED: src/utils/rawSource/index.js
      - Removed SourceIndexWriter import
      - Removed indexWriter instantiation in persist()
      - Removed indexWriter.append() call block
      - Removed re-export
      - Added `first_landing: written` to manifestRow
      - Updated JSDoc to reflect removal

Module count: 7 → 6 (SourceIndexWriter removed; 5 active + 1 stub)
Net LOC: -70 (40 LOC module + 30 LOC test removed)

Exports verified: createRawSourceService, createSourceStorage, ChecksumError,
createManifestWriter, createEmbeddingDispatcher, hashSource, sha256, sanitize,
SANITIZER_PATTERNS. No createIndexWriter — correctly absent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Correction 1.1 commit 3/5.

Route paths changed:
  GET /api/raw-sources/:hash      → GET /api/sessions/:sid/raw-sources/:hash
  GET /api/raw-sources/:hash/meta → GET /api/sessions/:sid/raw-sources/:hash/meta

Session-scoped routes now validate sessionId via SESSION_ID_RE and
instantiate SourceStorage inline per request with:
  poolDir = path.join(REPORTS_DIR_ABS, sessionId, 'raw-sources')
Storage construction is zero-I/O so per-request cost is negligible.

Removed from claude-sdk-server.js:
  - SOURCES_POOL_DIR constant (global pool concept)
  - _rawSourceStorage singleton + getRawSourceStorage() lazy loader
  - _ChecksumError instance check → replaced with err?.name === 'ChecksumError'
    (avoids needing to hold a module-level reference)

Added:
  - getRawSourceMod(): lazy module import (cached) replacing per-pool singleton
  - sessionPoolDir(sessionId): path helper
  - X-Session-Id response header on body GET for audit traceability

agentStreamHandler.js:
  - createRawSourceService call simplified: dropped poolDir param,
    now only passes { sessionsRoot: reportsRoot } since poolDir is
    derived inside persist() per call.

hookSSEBridge.js:
  - raw_source_ready SSE event URL updated from /api/raw-sources/{hash}
    to /api/sessions/{sessionId}/raw-sources/{hash} so the frontend can
    fetch directly without needing to know the session ID separately.

Already session-scoped routes (no change required):
  GET /api/sessions/:sid/raw-sources            — manifest route (unchanged)
  GET /api/sessions/:sid/agents/:agent/sources  — per-agent manifest (unchanged)

Syntax-clean: all three files pass node --check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Correction 1.1 commit 4/5. Updated all test fixtures for per-session pool.

Key changes:
  - Factory: { sessionsRoot } only (no poolDir)
  - Pool paths: root/{sessionId}/raw-sources/{ab}/{cd}/ (not root/_sources/)
  - Cross-session: both sessions write (written:true, different paths)
  - first_landing flag asserted on manifest rows
  - SourceIndexWriter test suite already deleted in commit 2

165 tests pass, 9 suites, 531ms.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…+ smoke

Correction 1.1 commit 5/5.

Updated all documentation artifacts to reflect the per-session pool:
  - reports/_sources/ → reports/{session_id}/raw-sources/ (path references)
  - /api/raw-sources/:hash → /api/sessions/:sid/raw-sources/:hash (route refs)
  - "Global pool" → "Per-session pool" (terminology)

Files updated:
  - docs/pending-updates/observability-updates-april-26.md (planning doc)
  - docs/runbooks/wave-1-deploy.md (deploy runbook)
  - test/smoke/README.md (smoke test curl commands)

Note: observability-implementation-spec.md was not updated in this commit
because the Correction 1.1 section already added to the plan file serves
as the canonical per-session design reference. The spec's Wave 1 §1.1
sections describe the original global-pool design for historical context.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ow-list

Bug fix discovered during live testing: raw-source archive was not
firing because subagents use SDK built-in WebFetch/WebSearch tools
(not MCP fetch_document/exa_web_search) when EXA_WEB_TOOLS=false
(the default).

Root cause: RAW_SOURCE_TOOLS only matched 'fetch_document' and
'exa_web_search' — the MCP tool names. When EXA_WEB_TOOLS is false,
PostToolUse fires with tool_name='WebFetch'/'WebSearch' which did
not match the allow-list.

Fix: split into two sets:
  RAW_SOURCE_MCP_TOOLS — .includes() match for MCP-wrapped variants
  RAW_SOURCE_SDK_TOOLS — exact-match Set for SDK built-in tools

isRawSourceTool() now checks both. Archive fires regardless of which
web-tool configuration is active.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Same root cause as 3c40727 (WebFetch/WebSearch missing from allow-lists)
but in two additional files. Discovered via log analysis of the live
test session: 404 tool calls (348 WebSearch + 56 WebFetch) were invisible
to injection detection AND SLA telemetry.

sdkHooks.js (postToolUseHandler):
  - textContent extraction condition widened:
    BEFORE: tool_name?.includes('fetch_document') || includes('exa_web_search')
    AFTER:  isMcpWebTool || isSdkWebTool (Set('WebFetch','WebSearch'))
  - JSON.parse wrapped in inner try/catch (SDK tools return raw HTML, not
    JSON — parse throws expectedly; textContent still populated for
    injection detection + metric labeling)
  - Net effect: promptInjectionDetector now scans WebFetch/WebSearch
    responses. Previously textContent was null → detector never ran.

hookDBBridge.js (persistAuditEvent):
  - SLA_HYBRID_TOOLS expanded: + 'WebFetch', 'WebSearch'
  - Non-JSON handling: set default fetch_source BEFORE JSON parse attempt:
      SDK tools (WebFetch/WebSearch) → 'sdk_builtin' (default, kept on
        JSON.parse failure since raw HTML isn't JSON)
      MCP tools with _hybrid_metadata → actual source (exa/native/etc.)
      MCP tools without metadata → 'native'
  - Net effect: /api/analytics/sla/7day now captures SDK-tool calls as
    fetch_source='sdk_builtin'. Previously zero rows populated.

Live session stats (pre-fix):
  WebSearch: 348 calls, WebFetch: 56 calls — none archived, scanned, or
  SLA-tracked. Post-fix all 404 calls will be captured across all three
  observability surfaces (#3, #8, #13).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Correction 1.2: move raw-source capture from PostToolUse hook to the
SDK message stream's content_block_start handler.

Root cause (discovered via live testing + log analysis + SDK source audit):
  Server-side tools (WebFetch/WebSearch) return results as specialized
  content blocks (web_fetch_tool_result, web_search_tool_result) in the
  agentQuery() message stream. PostToolUse hook DOES fire for these tools
  (8 tool_failure events prove it), but tool_response.content[0].text is
  empty — the actual response body flows through the stream, not the hook.

  The SDK's agentQuery() yields these blocks as stream_event messages with
  event.type === 'content_block_start'. Confirmed via:
    - SDK type definitions (BetaWebFetchToolResultBlock in sdk.d.ts)
    - Existing production usage (promptEnhancer.js:165-177 reads
      web_search_tool_result blocks from the same API)
    - includePartialMessages: true (already set at line 288)

Implementation in agentStreamHandler.js (line ~386):
  Two else-if branches in the content_block_start handler:

  web_fetch_tool_result:
    - block.content.type === 'web_fetch' → successful fetch (has HTML body)
    - block.content.content = full HTML response body
    - block.content.url = source URL
    - block.content.status = HTTP status code
    - Error blocks (type !== 'web_fetch') filtered — no point archiving 403s
    - Fire-and-forget: ctx.rawSourceService.persist({...}).then(emit SSE).catch(warn)

  web_search_tool_result:
    - block.content = Array<SearchResultBlock>
    - JSON.stringify'd before persist (structured results, not HTML)
    - result_count included in SSE event

Both paths:
  - Flag-gated: featureFlags.RAW_SOURCE_ARCHIVE
  - Null-safe: ctx.rawSourceService?.persist (service already on ctx from line 183)
  - SSE emission: type='hook_event', hook='raw_source_ready' (same shape as
    the hookSSEBridge path; frontend addRaw(e) captures it automatically)
  - Console log for live observability during testing

hookSSEBridge.js PostToolUse raw-source block:
  - Updated comment to document it as FALLBACK for MCP tools (EXA_WEB_TOOLS=true)
  - Functionally unchanged — still inert for default config (WebFetch/WebSearch
    don't populate tool_response.content[0].text)
  - Kept because MCP tools (fetch_document/exa_web_search) DO populate that
    field, so the hookSSEBridge path would work when EXA_WEB_TOOLS=true

Verification:
  - Syntax clean (node --check)
  - 165 unit + integration tests pass in 635ms (modules untouched)
  - Live test: restart server → raw-source pool files should appear
    incrementally during subagent WebFetch/WebSearch calls

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move raw-source capture from PostToolUse/stream (both inert) to the MCP
tool execution layer — the ONLY point where our code sees raw responses.

Root cause recap (three failed interception points):
  1. PostToolUse hook: tool_response.content[0].text is empty for
     server-side tools (WebFetch/WebSearch executed by API internally)
  2. Stream content_block_start: SDK doesn't yield web_fetch_tool_result
     or web_search_tool_result blocks (confirmed: zero in 3 live tests)
  3. Both fail because WebFetch/WebSearch are API-internal server tools

The fix: capture at wrapWithConversation() in toolImplementations.js.
This function wraps ALL 163 MCP tool handlers. Every external API call
returns its response through this single middleware. With EXA_WEB_TOOLS=true,
WebFetch/WebSearch are replaced by MCP tools (fetch_document, exa_web_search),
routing ALL web activity through wrapWithConversation. Coverage: 99.4%.

toolImplementations.js changes:
  - Added imports: path, fileURLToPath, getStore (requestContext),
    featureFlags, createRawSourceService
  - Added lazy singleton getRawSourceService() with __dirname-derived
    reportsRoot (zero I/O at import time; instantiates on first persist)
  - Inside wrapWithConversation: after tool execution + conversation logging,
    if RAW_SOURCE_ARCHIVE=true AND getStore()?.sessionDir is set:
      - Extract content: prefer MCP text field, fall back to JSON.stringify
      - Fire-and-forget persist({sessionId, toolName, content, url})
      - .catch() swallows errors; never breaks tool execution
      - Outer try/catch as belt-and-suspenders

agentStreamHandler.js changes:
  - Removed dead Path C code (web_fetch_tool_result / web_search_tool_result
    branches from commit 82bdc20) — confirmed these block types are never
    yielded by the SDK at runtime despite existing in type definitions
  - Replaced with comment explaining why and pointing to Correction 1.3

All 165 existing tests pass (rawSource modules untouched; only trigger moved).

Live test: restart with RAW_SOURCE_ARCHIVE=true EXA_WEB_TOOLS=true
  → MCP tool responses should populate reports/{sid}/raw-sources/

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SLA dashboard showed 'unknown' as the API client for all 248 tool calls
because SLA_HYBRID_TOOLS.has(tool_name) used exact match, but MCP tool
names arrive as 'mcp__super-legal-tools__fetch_document' (prefixed).
The set contained 'fetch_document' — exact match failed.

Fix: switch from .has() to .includes() pattern matching (same approach
hookSSEBridge already uses for isRawSourceTool). The isSlaTrackedTool
check now matches both exact names (WebFetch, WebSearch) and MCP-prefixed
variants (mcp__*__fetch_document, mcp__*__exa_web_search).

After fix, SLA dashboard will show:
  - 'native' or actual _hybrid_metadata.source for MCP tools with metadata
  - 'sdk_builtin' for SDK built-in tools (WebFetch/WebSearch)
  - Actual source values (exa, direct_fetch, etc.) when _hybrid_metadata parsed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Changelog entry for the Wave 1 institutional observability release:
  #3  raw-source archive (per-session, content-addressed, 287 sources live-tested)
  #8  prompt-injection detection (regex, logging-only)
  #12 per-tool latency histograms (P50/P95/P99)
  #13 7-day SLA dashboard per external API

Documents architecture corrections 1.1-1.3 discovered during live testing,
new files inventory, modified files summary, deployment notes, and flag
requirements (RAW_SOURCE_ARCHIVE requires EXA_WEB_TOOLS=true).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Number531 added a commit that referenced this pull request May 12, 2026
…validation) (#121)

Reframes documentation from "Exa as conditional feature flag" to "Exa as
production-default", with citation-verifier A/B validation evidence.

Drivers:
- EXA_WEB_TOOLS=true production-locked since 2026-04-18 (PR #76)
- Production-fidelity A/B validation 2026-05-12 (PRs #118 + #119):
  Exa 96.8% vs Anthropic 96.1% on 467-footnote citation-verifier
  fixture; both PASS production gate
- EXA_ADDITIONAL_QUERIES=true all-treatment in production flags.env
  since 2026-05-11 (v7.6.2)

Files (10):
- system-design.md — 6 edits: tool inventory framing, footnote groups,
  feature flag table, citation-verification roadmap row
- gtm-positioning-strategy.md — adds verifier validation metric
- gtm-sales-playbook.md — Q1 EU AI Act response cites Exa verifier
- gtm-buyer-intelligence.md — Q#8 cites Exa A/B validation
- enterprise-necessities.md — EXA flag entries reframed + new A3 entries
- feature-flags.md §26 — EXA_WEB_TOOLS production activation + validation
- runbooks/exa-a3-ab-staging.md — production rollout addendum (was
  "do NOT set in production yet")
- skills/client-audit-export — clarifies zero-rows interpretation by
  flag state (prevents false-positive incident filing)
- skills/deploy — post-deploy flag-verification block
- skills/subagent-scaffold — --a3-eligible reframed as RECOMMENDED

No code changes. No flag flips. Pure documentation alignment with
already-shipped production state.
Number531 added a commit that referenced this pull request May 22, 2026
Schema bump v1.1.0 -> v1.2.0. Pure additive enhancement to sidecar
projection — captures the agent's narrative `text` blocks alongside
the existing `thinking_summary` (reasoning blocks).

Why
---
thinking_summary captures the model's private reasoning. text_summary
captures what the agent says out loud between tool calls — stated
conclusions, plans, and observations. Operators triaging a session via
sidecar previously had to grep .full.jsonl to understand what the agent
concluded; now both reasoning streams live in the structured projection.

What changed
------------
- src/wrappedSubagents/transcriptSidecar.js:
  - SIDECAR_SCHEMA_VERSION 1.1.0 -> 1.2.0
  - New TEXT_SAMPLE_CHARS constant (300, matches THINKING_SAMPLE_CHARS)
  - New extractTextSummary() mirroring extractThinkingSummary (first +
    middle + last sample, total_blocks, total_chars)
  - buildSidecar() projects text_summary alongside thinking_summary

- test/sdk/wrappedSubagents/transcriptSidecar.test.js:
  - SIDECAR_SCHEMA_VERSION assertion bumped to '1.2.0' (2 sites)
  - text_summary added to required-keys assertion
  - +7 unit tests covering: block counting, empty turnLog, first+middle+
    last sampling, 300-char truncation, malformed block defensive handling,
    separation-of-concerns vs thinking blocks, single-block edge case

Validation
----------
- 41/41 transcriptSidecar tests pass (was 34; +7 new)
- 691/693 wrapped-subagents suite passes (2 failures pre-existing from
  Task #67 OPUS_MODEL fixture lag — unrelated)
- node --check passes
- Zero token cost (write-side projection only)
- Backward-compatible: older consumers ignore the new field

Plan: docs/pending-updates/Wrapped-Migration-Phase4.13-Full-Fidelity-Transcripts.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant